Pandas Profiling Report

Dataset statistics

Number of variables	10
Number of observations	14609
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	32
Duplicate rows (%)	0.2%
Total size in memory	1.1 MiB
Average record size in memory	80.0 B

Variable types

Numeric	10

Alerts

Dataset has 32 (0.2%) duplicate rows	Duplicates
`Fwd Pkt Len Mean` is highly overall correlated with `Flow Duration` and 4 other fields	High correlation
`Fwd IAT Mean` is highly overall correlated with `Pkt Len Var` and 2 other fields	High correlation
`Pkt Len Mean` is highly overall correlated with `Flow Duration` and 4 other fields	High correlation
`Pkt Len Var` is highly overall correlated with `Fwd IAT Mean`	High correlation
`Pkt Size Avg` is highly overall correlated with `Flow Duration` and 6 other fields	High correlation
`Fwd Seg Size Avg` is highly overall correlated with `Flow Duration` and 4 other fields	High correlation
`Bwd Seg Size Avg` is highly overall correlated with `Flow Duration` and 4 other fields	High correlation
`Active Mean` is highly overall correlated with `Pkt Len Var`	High correlation
`Idle Mean` is highly overall correlated with `Flow Duration` and 2 other fields	High correlation
`Flow Duration` is highly overall correlated with `Fwd Pkt Len Mean` and 5 other fields	High correlation
`Pkt Len Var` has 218 (1.5%) zeros	Zeros
`Active Mean` has 699 (4.8%) zeros	Zeros
`Idle Mean` has 599 (4.1%) zeros	Zeros

Reproduction

Analysis started	2022-12-12 12:36:21.531731
Analysis finished	2022-12-12 12:36:37.122526
Duration	15.59 seconds
Software version	pandas-profiling vv3.5.0
Download configuration	config.json

Flow Duration
Real number (ℝ)

Distinct	8346
Distinct (%)	57.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	1.0176367 × 10⁸

Minimum	172154.14
Maximum	1.2 × 10⁸
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	172154.14
5-th percentile	8810872.7
Q1	1.0928317 × 10⁸
median	1.1370661 × 10⁸
Q3	1.1573552 × 10⁸
95-th percentile	1.1828162 × 10⁸
Maximum	1.2 × 10⁸
Range	1.1982785 × 10⁸
Interquartile range (IQR)	6452352.1

Descriptive statistics

Standard deviation	32759923
Coefficient of variation (CV)	0.3219216
Kurtosis	4.1111453
Mean	1.0176367 × 10⁸
Median Absolute Deviation (MAD)	2189885.6
Skewness	-2.4393706
Sum	1.4866654 × 10¹²
Variance	1.0732126 × 10¹⁵
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
8810872.65	661	4.5%
113706608.1	638	4.4%
111753381.8	637	4.4%
106772664.3	637	4.4%
115860880.9	615	4.2%
115232186.8	614	4.2%
105644629.1	446	3.1%
114162049.9	373	2.6%
11557810.88	373	2.6%
116887717.4	359	2.5%
Other values (8336)	9256	63.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
172154.1429	1	< 0.1%
191767.1111	1	< 0.1%
194978.125	1	< 0.1%
195003.25	1	< 0.1%
198201.75	1	< 0.1%
198345	1	< 0.1%
199043.0909	1	< 0.1%
201672.7333	1	< 0.1%
202731.4615	1	< 0.1%
203114.4545	1	< 0.1%

Value	Count	Frequency (%)
120000000	1	< 0.1%
119997450	1	< 0.1%
119995846	1	< 0.1%
119991605	1	< 0.1%
119991198	1	< 0.1%
119989397	1	< 0.1%
119986342	1	< 0.1%
119984575	1	< 0.1%
119979525.5	1	< 0.1%
119977985.8	1	< 0.1%

Fwd Pkt Len Mean
Real number (ℝ)

Distinct	8315
Distinct (%)	56.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	293.87977

Minimum	0
Maximum	1453.6481
Zeros	1
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	66.925511
Q1	207.61419
median	274.00105
Q3	366.62028
95-th percentile	606.59422
Maximum	1453.6481
Range	1453.6481
Interquartile range (IQR)	159.00609

Descriptive statistics

Standard deviation	148.45261
Coefficient of variation (CV)	0.50514744
Kurtosis	1.4505709
Mean	293.87977
Median Absolute Deviation (MAD)	78.895791
Skewness	0.6712408
Sum	4293289.6
Variance	22038.179
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
509.516994	661	4.5%
236.6956492	638	4.4%
84.69290836	637	4.4%
66.92551068	637	4.4%
606.5942162	615	4.2%
237.9919745	614	4.2%
435.628899	446	3.1%
318.2903519	373	2.6%
352.8968374	373	2.6%
207.6141922	359	2.5%
Other values (8305)	9256	63.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	1	< 0.1%
12	2	< 0.1%
13.06593407	1	< 0.1%
21.88783069	1	< 0.1%
22.88549619	1	< 0.1%
28.77553422	1	< 0.1%
29.89917899	1	< 0.1%
30.92392638	1	< 0.1%
32.93133803	1	< 0.1%
32.95178572	1	< 0.1%

Value	Count	Frequency (%)
1453.648116	1	< 0.1%
1453.282297	1	< 0.1%
1451.847875	1	< 0.1%
1448.992526	1	< 0.1%
1447.717813	1	< 0.1%
1446.958932	1	< 0.1%
1094.736141	1	< 0.1%
994.3990847	1	< 0.1%
989.350973	1	< 0.1%
985.3638437	1	< 0.1%

Fwd IAT Mean
Real number (ℝ)

Distinct	8306
Distinct (%)	56.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	3163251.8

Minimum	0
Maximum	18700000
Zeros	33
Zeros (%)	0.2%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	484820.57
Q1	1554344.6
median	3174253.3
Q3	4252937.5
95-th percentile	6420068.6
Maximum	18700000
Range	18700000
Interquartile range (IQR)	2698592.9

Descriptive statistics

Standard deviation	1856436.2
Coefficient of variation (CV)	0.58687589
Kurtosis	2.711012
Mean	3163251.8
Median Absolute Deviation (MAD)	1229664.7
Skewness	0.79870426
Sum	4.6211946 × 10¹⁰
Variance	3.4463555 × 10¹²
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
484820.5699	661	4.5%
1554344.596	638	4.4%
4628602.588	637	4.4%
2835132.453	637	4.4%
1247332.517	615	4.2%
4001714.419	614	4.2%
7304310.57	446	3.1%
5323309.353	373	2.6%
71865.17121	373	2.6%
2164658.695	359	2.5%
Other values (8296)	9256	63.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	33	0.2%
71	1	< 0.1%
248.8726708	1	< 0.1%
453.2105263	1	< 0.1%
747.3846154	1	< 0.1%
952.3636364	1	< 0.1%
1588.642857	1	< 0.1%
1755.796297	1	< 0.1%
4929.447368	1	< 0.1%
5657.392225	1	< 0.1%

Value	Count	Frequency (%)
18700000	1	< 0.1%
18000000	1	< 0.1%
17800000	1	< 0.1%
16200000	1	< 0.1%
15000000	9	0.1%
14200000	1	< 0.1%
13333333.33	1	< 0.1%
13333258.73	1	< 0.1%
13251048.25	1	< 0.1%
13093161.1	1	< 0.1%

Pkt Len Mean
Real number (ℝ)

Distinct	8315
Distinct (%)	56.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	292.90475

Minimum	0
Maximum	1453.659
Zeros	1
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	70.741203
Q1	207.61226
median	272.89602
Q3	365.58279
95-th percentile	606.62705
Maximum	1453.659
Range	1453.659
Interquartile range (IQR)	157.97053

Descriptive statistics

Standard deviation	148.31746
Coefficient of variation (CV)	0.50636755
Kurtosis	1.4723088
Mean	292.90475
Median Absolute Deviation (MAD)	80.344308
Skewness	0.69013286
Sum	4279045.5
Variance	21998.069
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
509.5214262	661	4.5%
236.6341502	638	4.4%
84.81351348	637	4.4%
70.74120257	637	4.4%
606.6270486	615	4.2%
232.4193871	614	4.2%
429.0506408	446	3.1%
315.4352585	373	2.6%
353.2403274	373	2.6%
207.6122616	359	2.5%
Other values (8305)	9256	63.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	1	< 0.1%
12	2	< 0.1%
13.17142857	1	< 0.1%
21.87695516	1	< 0.1%
22.85714286	1	< 0.1%
28.79294723	1	< 0.1%
29.90596072	1	< 0.1%
30.94002448	1	< 0.1%
32.9493007	1	< 0.1%
32.96631206	1	< 0.1%

Value	Count	Frequency (%)
1453.658974	1	< 0.1%
1453.298329	1	< 0.1%
1451.884187	1	< 0.1%
1449.025335	1	< 0.1%
1447.760984	1	< 0.1%
1447.01227	1	< 0.1%
1094.74337	1	< 0.1%
994.1877578	1	< 0.1%
989.3419161	1	< 0.1%
985.4146134	1	< 0.1%

Pkt Len Var
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	8243
Distinct (%)	56.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	2765.409

Minimum	0
Maximum	57366.062
Zeros	218
Zeros (%)	1.5%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	81.800395
Q1	1159.7846
median	2831.2342
Q3	3597.5717
95-th percentile	6310.6169
Maximum	57366.062
Range	57366.062
Interquartile range (IQR)	2437.7871

Descriptive statistics

Standard deviation	2424.6645
Coefficient of variation (CV)	0.8767833
Kurtosis	63.841706
Mean	2765.409
Median Absolute Deviation (MAD)	1363.1487
Skewness	4.4920839
Sum	40399860
Variance	5878997.7
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
190.1616446	661	4.5%
1187.088024	638	4.4%
3579.009639	637	4.4%
357.670288	637	4.4%
2990.705458	615	4.2%
6012.953024	614	4.2%
7903.578068	446	3.1%
3352.507692	373	2.6%
81.80039475	373	2.6%
1721.908717	359	2.5%
Other values (8233)	9256	63.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	218	1.5%
0.008297586	1	< 0.1%
0.016666667	1	< 0.1%
0.023581067	1	< 0.1%
0.048076923	1	< 0.1%
0.052067697	1	< 0.1%
0.063186813	1	< 0.1%
0.080132451	1	< 0.1%
0.082417582	1	< 0.1%
0.083333333	1	< 0.1%

Value	Count	Frequency (%)
57366.06247	1	< 0.1%
46694.44444	2	< 0.1%
45395.73773	1	< 0.1%
45026.78571	2	< 0.1%
38564.85955	1	< 0.1%
31498.16113	1	< 0.1%
27645.23103	1	< 0.1%
27634.05654	1	< 0.1%
27230.98488	1	< 0.1%
26781.56183	1	< 0.1%

Pkt Size Avg
Real number (ℝ)

Distinct	8322
Distinct (%)	57.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	322.14837

Minimum	0
Maximum	1455.125
Zeros	1
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	74.51799
Q1	211.76099
median	279.26893
Q3	399.75337
95-th percentile	741.88074
Maximum	1455.125
Range	1455.125
Interquartile range (IQR)	187.99239

Descriptive statistics

Standard deviation	189.56092
Coefficient of variation (CV)	0.58842737
Kurtosis	0.84974786
Mean	322.14837
Median Absolute Deviation (MAD)	81.35427
Skewness	1.0113427
Sum	4706265.6
Variance	35933.343
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
741.8807375	661	4.5%
237.8101539	638	4.4%
86.81375316	637	4.4%
74.51798978	637	4.4%
608.358666	615	4.2%
236.4514086	614	4.2%
435.6487973	446	3.1%
524.0790918	373	2.6%
328.4370947	373	2.6%
208.8986524	359	2.5%
Other values (8312)	9256	63.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	1	< 0.1%
12.1	1	< 0.1%
12.10084034	1	< 0.1%
14.30498866	1	< 0.1%
23.03030303	1	< 0.1%
23.26830732	1	< 0.1%
30.12226602	1	< 0.1%
30.64673227	1	< 0.1%
30.97794118	1	< 0.1%
33.06491228	1	< 0.1%

Value	Count	Frequency (%)
1455.125	1	< 0.1%
1455.034648	1	< 0.1%
1454.902481	1	< 0.1%
1451.18806	1	< 0.1%
1450.309859	1	< 0.1%
1449.977459	1	< 0.1%
1141.534762	1	< 0.1%
1081.8	1	< 0.1%
1037.773477	1	< 0.1%
1037.500333	1	< 0.1%

Fwd Seg Size Avg
Real number (ℝ)

Distinct	8315
Distinct (%)	56.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	293.87977

Minimum	0
Maximum	1453.6481
Zeros	1
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	66.925511
Q1	207.61419
median	274.00105
Q3	366.62028
95-th percentile	606.59422
Maximum	1453.6481
Range	1453.6481
Interquartile range (IQR)	159.00609

Descriptive statistics

Standard deviation	148.45261
Coefficient of variation (CV)	0.50514744
Kurtosis	1.4505709
Mean	293.87977
Median Absolute Deviation (MAD)	78.895791
Skewness	0.6712408
Sum	4293289.6
Variance	22038.179
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
509.516994	661	4.5%
236.6956492	638	4.4%
84.69290836	637	4.4%
66.92551068	637	4.4%
606.5942162	615	4.2%
237.9919745	614	4.2%
435.628899	446	3.1%
318.2903519	373	2.6%
352.8968374	373	2.6%
207.6141922	359	2.5%
Other values (8305)	9256	63.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	1	< 0.1%
12	2	< 0.1%
13.06593407	1	< 0.1%
21.88783069	1	< 0.1%
22.88549619	1	< 0.1%
28.77553422	1	< 0.1%
29.89917899	1	< 0.1%
30.92392638	1	< 0.1%
32.93133803	1	< 0.1%
32.95178572	1	< 0.1%

Value	Count	Frequency (%)
1453.648116	1	< 0.1%
1453.282297	1	< 0.1%
1451.847875	1	< 0.1%
1448.992526	1	< 0.1%
1447.717813	1	< 0.1%
1446.958932	1	< 0.1%
1094.736141	1	< 0.1%
994.3990847	1	< 0.1%
989.350973	1	< 0.1%
985.3638437	1	< 0.1%

Bwd Seg Size Avg
Real number (ℝ)

Distinct	7111
Distinct (%)	48.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	293.13272

Minimum	0
Maximum	1460
Zeros	1
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	89.945455
Q1	210.16667
median	273.69231
Q3	367.57895
95-th percentile	610.5
Maximum	1460
Range	1460
Interquartile range (IQR)	157.41228

Descriptive statistics

Standard deviation	147.3605
Coefficient of variation (CV)	0.50270916
Kurtosis	1.6346008
Mean	293.13272
Median Absolute Deviation (MAD)	77.307692
Skewness	0.76221544
Sum	4282375.9
Variance	21715.118
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
510	662	4.5%
239.4705882	638	4.4%
90.28571429	637	4.4%
93.22222222	637	4.4%
610.5	615	4.2%
217.3333333	614	4.2%
407.625	446	3.1%
307.4444444	373	2.6%
351	373	2.6%
213.7272727	359	2.5%
Other values (7101)	9255	63.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	1	< 0.1%
12	2	< 0.1%
13.85714286	1	< 0.1%
21	1	< 0.1%
21.14285714	1	< 0.1%
22	4	< 0.1%
24.54545455	1	< 0.1%
27.33333333	1	< 0.1%
30.28571429	1	< 0.1%
32	1	< 0.1%

Value	Count	Frequency (%)
1460	6	< 0.1%
1098	1	< 0.1%
998.3333333	1	< 0.1%
996.3333333	1	< 0.1%
990.6666667	1	< 0.1%
977.3333333	4	< 0.1%
890	1	< 0.1%
886.8	2	< 0.1%
855.4285714	1	< 0.1%
817.5	1	< 0.1%

Active Mean
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	8053
Distinct (%)	55.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	615272.1

Minimum	0
Maximum	24550000
Zeros	699
Zeros (%)	4.8%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	991.43647
Q1	77919.63
median	239639.84
Q3	576768.97
95-th percentile	3058145.8
Maximum	24550000
Range	24550000
Interquartile range (IQR)	498849.34

Descriptive statistics

Standard deviation	1278281.3
Coefficient of variation (CV)	2.077587
Kurtosis	52.241469
Mean	615272.1
Median Absolute Deviation (MAD)	184845.98
Skewness	5.60469
Sum	8.9885101 × 10⁹
Variance	1.6340031 × 10¹²
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	699	4.8%
48424.725	661	4.5%
124190.6005	638	4.4%
77919.62963	637	4.4%
371114.7381	637	4.4%
225047.4167	615	4.2%
626362.2639	614	4.2%
622491.6198	446	3.1%
767778.4931	373	2.6%
26360.94318	359	2.5%
Other values (8043)	8930	61.1%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	699	4.8%
5.411764706	1	< 0.1%
10.1	1	< 0.1%
12.57142857	1	< 0.1%
20.84	1	< 0.1%
23.81818182	1	< 0.1%
28.83333333	1	< 0.1%
35.71428571	1	< 0.1%
36.07692308	1	< 0.1%
37.33333333	1	< 0.1%

Value	Count	Frequency (%)
24550000	1	< 0.1%
22985558.5	1	< 0.1%
22825000	1	< 0.1%
21634217	1	< 0.1%
18307280.75	1	< 0.1%
17250000	1	< 0.1%
16995773.94	1	< 0.1%
15964244.73	1	< 0.1%
15819461.75	1	< 0.1%
15350000	1	< 0.1%

Idle Mean
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct	7254
Distinct (%)	49.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	10422189

Minimum	0
Maximum	90000000
Zeros	599
Zeros (%)	4.1%
Negative	0
Negative (%)	0.0%
Memory size	114.3 KiB

Quantile statistics

Minimum	0
5-th percentile	1471855.5
Q1	7673684.2
median	10804545
Q3	13255556
95-th percentile	18733697
Maximum	90000000
Range	90000000
Interquartile range (IQR)	5581871.3

Descriptive statistics

Standard deviation	5863050.6
Coefficient of variation (CV)	0.56255461
Kurtosis	7.0577517
Mean	10422189
Median Absolute Deviation (MAD)	2586897.9
Skewness	1.4552386
Sum	1.5225776 × 10¹¹
Variance	3.4375363 × 10¹³
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
2429015.506	661	4.5%
10958811.67	638	4.4%
13255555.56	638	4.4%
11442857.08	637	4.4%
9137500	616	4.2%
13588888.89	614	4.2%
0	599	4.1%
30562500	446	3.1%
13522219.51	373	2.6%
10881818.18	359	2.5%
Other values (7244)	9028	61.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	599	4.1%
329924.8333	1	< 0.1%
450000	1	< 0.1%
456250	1	< 0.1%
480952.381	1	< 0.1%
513994.8553	1	< 0.1%
541990.3333	1	< 0.1%
600000	1	< 0.1%
610361.4531	1	< 0.1%
617391.3043	1	< 0.1%

Value	Count	Frequency (%)
90000000	1	< 0.1%
72000000	1	< 0.1%
59950000	1	< 0.1%
49633333.33	1	< 0.1%
45000000	1	< 0.1%
44133333.33	1	< 0.1%
42466592.18	1	< 0.1%
40000000	1	< 0.1%
39733333.33	1	< 0.1%
39566666.67	1	< 0.1%

Auto

The auto setting is an interpretable pairwise column metric of the following mapping:

Variable_type-Variable_type : Method, Range
Categorical-Categorical : Cramer's V, [0,1]
Numerical-Categorical : Cramer's V, [0,1] (using a discretized numerical column)
Numerical-Numerical : Spearman's ρ, [-1,1]

The number of bins used in the discretization for the Numerical-Categorical column pair can be changed using config.correlations["auto"].n_bins. The number of bins affects the granularity of the association you wish to measure.

This configuration uses the recommended metric for each pair of columns.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Count
Matrix

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

First rows
Last rows

	Flow Duration	Fwd Pkt Len Mean	Fwd IAT Mean	Pkt Len Mean	Pkt Len Var	Pkt Size Avg	Fwd Seg Size Avg	Bwd Seg Size Avg
0	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
1	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
2	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
3	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
4	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
5	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
6	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
7	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
8	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0
9	11557810.88	352.896837	71865.17121	353.240327	81.800395	524.079092	352.896837	351.0

	Flow Duration	Fwd Pkt Len Mean	Fwd IAT Mean	Pkt Len Mean	Pkt Len Var	Pkt Size Avg	Fwd Seg Size Avg	Bwd Seg Size Avg	Active Mean	Idle Mean
14599	2.892786e+05	420.125000	0.000000	420.125000	0.000000	630.187500	420.125000	420.125000	0.00000	0.000
14600	6.034903e+06	638.155891	6307.749163	638.156757	252.213326	927.056103	638.155891	638.458333	19958.20833	2050000.000
14601	2.599031e+05	516.555556	0.000000	516.555556	0.000000	774.833333	516.555556	516.555556	0.00000	0.000
14602	3.636731e+06	613.078743	643641.331200	613.082986	23.394605	908.071020	613.078743	613.333333	0.00000	625000.000
14603	1.917671e+05	512.888889	1755.796297	512.888889	0.000000	769.333333	512.888889	512.888889	0.00000	0.000
14604	2.212897e+07	477.334387	713692.217300	475.273352	1988.924082	699.971666	477.334387	469.040000	319135.18000	1449650.650
14605	6.624616e+06	546.666667	229044.261800	546.666667	0.000000	820.000000	546.666667	546.666667	0.00000	0.000
14606	4.334327e+06	497.346154	275173.718000	496.961539	2.884615	744.079487	497.346154	496.769231	15049.52885	2534549.375
14607	6.107274e+06	472.000000	488122.559600	472.000000	0.000000	708.000000	472.000000	472.000000	71239.64286	1387979.893
14608	7.698330e+06	527.006937	994409.903000	527.027483	435.623800	759.971744	527.006937	527.884615	33627.97115	1296990.404

Most frequently occurring

	Flow Duration	Fwd Pkt Len Mean	Fwd IAT Mean	Pkt Len Mean	Pkt Len Var	Pkt Size Avg	Fwd Seg Size Avg	Bwd Seg Size Avg	Active Mean	Idle Mean	# duplicates
2	8.810873e+06	509.516994	4.848206e+05	509.521426	190.161645	741.880738	509.516994	510.000000	48424.72500	2.429016e+06	661
14	1.137066e+08	236.695649	1.554345e+06	236.634150	1187.088024	237.810154	236.695649	239.470588	124190.60050	1.095881e+07	638
8	1.067727e+08	84.692908	2.835132e+06	84.813513	357.670288	86.813753	84.692908	93.222222	77919.62963	1.325556e+07	637
11	1.117534e+08	66.925511	4.628603e+06	70.741203	3579.009639	74.517990	66.925511	90.285714	371114.73810	1.144286e+07	637
26	1.158609e+08	606.594216	1.247333e+06	606.627049	2990.705458	608.358666	606.594216	610.500000	225047.41670	9.137500e+06	615
19	1.152322e+08	237.991974	4.001714e+06	232.419387	6012.953024	236.451409	237.991974	217.333333	626362.26390	1.358889e+07	614
7	1.056446e+08	435.628899	7.304311e+06	429.050641	7903.578068	435.648797	435.628899	407.625000	622491.61980	3.056250e+07	446
3	1.155781e+07	352.896837	7.186517e+04	353.240327	81.800395	524.079092	352.896837	351.000000	0.00000	0.000000e+00	373
15	1.141620e+08	318.290352	5.323309e+06	315.435258	3352.507692	328.437095	318.290352	307.444444	767778.49310	1.352222e+07	373
29	1.168877e+08	207.614192	2.164659e+06	207.612262	1721.908717	208.898652	207.614192	213.727273	26360.94318	1.088182e+07	359

Overview

Variables

Interactions

Correlations

Auto

Spearman's ρ

Pearson's r

Kendall's τ

Phik (φk)

Missing values

Sample

Duplicate rows

Most frequently occurring